Skip to content

Fix conversion options validation failing on type and callable objects#1667

Merged
bendichter merged 6 commits intomainfrom
fix-type-json-serialization
Feb 18, 2026
Merged

Fix conversion options validation failing on type and callable objects#1667
bendichter merged 6 commits intomainfrom
fix-type-json-serialization

Conversation

@bendichter
Copy link
Contributor

@bendichter bendichter commented Feb 13, 2026

Summary

  • validate_conversion_options was updated in Support datetime in conversion options #1139 to JSON-encode conversion_options before schema validation. This broke passing progress_bar_class (a type object) and callback functions in progress_bar_options, as _NWBConversionOptionsEncoder had no handler for these types.
  • Add handling for type and callable objects in _NWBConversionOptionsEncoder, serializing them to their qualified module path string for validation purposes. The original objects are still passed through to the actual conversion code unchanged.

How this manifests

NWB GUIDE sets progress_bar_class=TQDMProgressSubscriber (a class, not an instance) in iterator_opts within conversion_options at manage_neuroconv.py:1021:

options_to_update["iterator_opts"] = dict(
    display_progress=True,
    progress_bar_class=TQDMProgressSubscriber,  # <-- type object
    progress_bar_options=progress_bar_options,   # <-- contains a callable
)

This dict ends up in conversion_options, which is passed to NWBConverter.run_conversion(). Inside run_conversion (nwbconverter.py:319):

self.validate_conversion_options(conversion_options=conversion_options)

Which calls (nwbconverter.py:201):

encoded_conversion_options = _NWBConversionOptionsEncoder().encode(conversion_options)

The encoder has no handler for type or callable objects, so it falls through to json.JSONEncoder.default() which raises:

TypeError: Object of type type is not JSON serializable

At runtime (after validation), the conversion_options dict is unpacked as kwargs into each interface's add_to_nwbfile() (nwbconverter.py:253):

data_interface.add_to_nwbfile(nwbfile=nwbfile, metadata=metadata, **conversion_options.get(interface_name, dict()))

This passes iterator_opts as a kwarg, which flows through to SpikeInterfaceRecordingDataChunkIterator / ImagingExtractorDataChunkIterator, then to hdmf's GenericDataChunkIterator, where progress_bar_class is used as a constructor. This runtime path works fine — the issue is only in the validation step.

Before vs after #1139

Before #1139, validation was:

validate(instance=conversion_options or {}, schema=self.get_conversion_options_schema())

jsonschema.validate() traverses the dict but doesn't serialize it, so class objects were fine.

After #1139:

encoded = _NWBConversionOptionsEncoder().encode(conversion_options)
decoded = json.loads(encoded)
validate(instance=decoded, schema=self.get_conversion_options_schema())

The JSON encode/decode round-trip fails on non-serializable types.

Test plan

  • Verify that passing progress_bar_class as a type object in conversion_options no longer raises TypeError: Object of type type is not JSON serializable
  • Verify that passing callable objects in progress_bar_options works
  • Verify that normal conversion options (strings, bools, dicts) still validate correctly

🤖 Generated with Claude Code

`validate_conversion_options` was updated in #1139 to JSON-encode
conversion_options before schema validation. This broke passing
`progress_bar_class` (a type) and callback functions in
`progress_bar_options`, as the encoder had no handler for these types.

Add handling for `type` and `callable` objects in
`_NWBConversionOptionsEncoder`, serializing them to their qualified
module path for validation purposes. The original objects are still
passed through to the actual conversion code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@bendichter
Copy link
Contributor Author

@h-mayorquin can you please look at this when you get a chance?

Copy link
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a test and a changelog but LGTM.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@bendichter bendichter merged commit 91ddbb1 into main Feb 18, 2026
23 of 24 checks passed
@bendichter bendichter deleted the fix-type-json-serialization branch February 18, 2026 14:25
@codecov
Copy link

codecov bot commented Feb 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.22%. Comparing base (7523a15) to head (aa9d40f).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1667   +/-   ##
=======================================
  Coverage   80.22%   80.22%           
=======================================
  Files         160      160           
  Lines       12655    12659    +4     
=======================================
+ Hits        10152    10156    +4     
  Misses       2503     2503           
Flag Coverage Δ
unittests 80.22% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/neuroconv/utils/json_schema.py 96.34% <100.00%> (+0.09%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants